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Structural  Understanding  in  Problem  Solving: 
Specifying  and  Instructing  Cognitive  Objectives 
in  Basic  Electricity 


Overview 


~  A  common  instructional  objective  in  domains  of  math  and  science  is  the  capability  to  use  formu¬ 
las  and  arithmetic  procedures  to  solve  problems.  Although  students  are  explicitly  taught  the  relevant 
formulas  and  principles,  are  shown  worked-out  examples,  and  are  given  practice,  they  frequently 
experience  considerable  difficulty  when  asked  to  solve  similar  problems.  Previous  research -{Rileys 
1984)^ suggested  that  difficulties  often  result  from  mechanical  application  of  rules  and  formulas  with 
little  understanding  of  important  structural  relations  between  the  elements  in  the  problem  domain. 
The  objective  of  research  is  to  analyze  in  greater  detail  what  is  meant  by  structural  understanding 
within  the  domain  of  basic  electricity,  the  role  this  understanding  plays  in  performance  and  learning, 
and  the  extent  to  which  important  structural  relations  can  be  taught  more  directly.  Research  has 
involved  (1)  empirical  studies  of  the  knowledge  students  acquire  (and  fail  to  acquire)  both  from  the 
Navy’s  Basic  Electricity  and  Electronics  (BE&E)  course  material  and  from  alternative  instructional 
materials  and  (2)  theoretical  analyses  in  the  form  of  computer  simulation  models  that  represent 
detailed  hypotheses  about  the  knowledge  underlying  students’  performance  and  learning. 
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Group  Members 

Mary  Riley  (Principal  Investigator),  Eric  Hcstenes  (Programmer),  Liz  Troschinetz  and  Susan 
Voss  (Research  Personnel). 

Eva  Lee  man,  Robert  Moore,  and  Lori  Prince  joined  the  group  for  the  Spring  quarter  to  work  on 
senior  projects  concerning  issues  of  learning  and  performance  in  basic  electricity.  (See  the  section 
entitled  'Unpublished  Manuscripts'.) 


Empirical  Studies 

We  are  conducting  two  main  kinds  of  empirical  studies:  protocol  studies  of  individual  subjects 
and  group  studies.  The  goal  of  the  protocol  studies  is  to  provide  detailed  information  about  the 
knowledge  underlying  problem-solving  performance  and  learning,  €4.,  information  about  the  sequence 
and  types  of  solution  steps,  whether  the  subject  reason  using  equations  or  by  analogy,  whether  the 
subject  reason  using  quantitative  or  qualitative  relations.  This  information  is  important  in  the 
development  of  the  simulation  models  and  is  not  directly  available  from  tests  administered  to  large 
groups. 

In  the  group  studies,  I  am  collaborating  with  Dr.  William  E.  Montague,  Naval  Personnel 
Research  and  Development  Center,  to  obtain  performance  data  from  actual  BE&E  students  and  gra¬ 
duates  solving  paper-and-pencil  problems  like  the  ones  used  in  the  protocol  studies.  We  are  using  this 
data,  together  with  information  obtained  from  the  theoretical  analyses,  to  develop  diagnostic  tests  for 
monitoring  knowledge  and  skill  acquisition  in  BE&E.  The  group  data  also  provides  a  valuable  test  of 
the  generality  of  the  findings  from  the  protocol  studies. 

Protocol  Studies 

Reasoning  about  DC  Circuits  Using  Ohm's  and  Kirchhoffs  Laws 

During  April,  May,  and  June,  we  obtained  verbal  protocols  from  five  subjects  as  they  learned 
about  DC  circuits  from  an  abbreviated  version  of  the  standard  BE&E  course. 1  To  monitor  subjects’ 
understanding  as  they  progressed  through  the  course  material,  a  diagnostic  set  of  free-responsc  prob¬ 
lems  was  generated  to  replace  the  multiple-choice  problems  used  in  the  BE&E  course  materials.  In 
her  protocol  analysis  of  multiple  choice  problems,  Riley  (1984)  found  that  students  did  not  reason 
through  the  answers  to  determine  the  correct  choice.  Rather,  they  compared  patterns  of  answers 
eliminating  incorrect  alternatives.  In  order  to  avoid  this  problem,  we  developed  a  diagnostic  test  that 
would  force  students  to  reason  through  each  problem  and  to  indicate  more  directly  how  well  a  stu¬ 
dent  understood  a  problem. 


1.  We  bad  alio  planned  to  study  the  knowledge  students  acquire  about  altcrnsting-curreot  (AC)  circuit*  from  an  abbreviated 
version  of  tbc  BEJtE  course  material.  Subjects  were  to  have  been  tbe  sane  a*  those  who  studied  the  DC  circuits,  to  provide 
ioformatioa  about  bow  their  knowledge  of  DC  circuits  influences  their  ability  to  learn  about  AC  circuits.  However,  subjects' 
time  constraints  resulting  from  tbe  quartet  clast  schedule  prevented  their  continuing  beyond  tbe  instruction  on  DC  circuits. 
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A  major  focus  of  our  research  concerns  the  relationship  between  quantitative  and  qualitative 
reasoning  about  circuit  variables.  Therefore  the  diagnostic  test  included  both  quantitative  and  quali¬ 
tative  problems.  Quantitative  problems  give  the  value(s)  of  one  or  more  circuit  variables  (current,  vol¬ 
tage,  and/or  resistance)  and  ask  subjects  to  solve  for  the  value  of  another  circuit  variable.  Qualita¬ 
tive  problems  involve  reasoning  about  increases  and  decreases  in  circuit  variables  without  using  the 
actual  values  of  the  variables.  Qualitative  'Effect*  problems  require  the  subject  to  determine  the 
effects  of  changing  the  value  of  one  circuit  variable  on  other  circuit  variables  (do  they  increase, 
decrease,  or  remain  the  same?);  Qualitative  'Cause*  problems  require  identifying  the  set  of  possible 
events  that  could  have  caused  specific  changes  in  the  value  of  circuit  variables. 

Tasks  also  included  pretests  to  provide  specific  information  about  relevant  cognitive  abilities 
prior  to  instruction.  In  particular  we  were  interested  in  subjects*  understanding  of  isolated  equations 
involving  additive  and  multiplicative  relations.  These  are  the  same  relations  involved  in  Ohm’s  and 
Kirchhoffs  laws.  If  subjects  lacked  an  understanding  of  these  relations  in  isolated  equations,  we 
would  expect  them  to  have  difficulty  applying  several  equations  to  reason  about  simultaneous  con¬ 
straints  on  circuit  variables. 

Protocols  have  been  transcribed  and  coded  and  initial  analyses  have  been  completed.  Prelim¬ 
inary  results  indicate  that 

•  Subjects  could  solve  isolated  equations  when  numbers  were  provided  but  had  some  difficulty 
reasoning  about  qualitative  relations  between  variables  in  these  equations. 

•  Similarly,  subjects  could  reason  correctly  about  circuit  problems  when  numbers  were  provided 
but  made  frequent  errors  when  reasoning  about  qualitative  relations  between  current,  voltage, 
and  resistance. 

•  Pretest  performance  appears  to  be  correlated  with  performance  on  circuit  problems.  However, 
successful  performance  on  the  pretest  does  not  guarantee  successful  performance  on  circuit 
problems. 

•  These  data  support  and  extend  earlier  findings  (Riley,  1984). 


Learning  by  Analogy 

Our  studies  indicate  that  many  subjects  (and  BEAE  students)  attempt  to  memorize  algorithms 
for  manipulating  equations  to  solve  particular  problems,  without  understanding  the  relationships 
between  the  elements  in  the  equations,  let  alone  how  these  equations  relate  to  the  relations  between 
current,  voltage,  and  resistance  in  electrical  circuits.  We  are  exploring  teaching  subjects  important 
structural  relations  between  circuit  variables  using  a  concrete  analogy.  As  a  first  step,  we  are  analyz¬ 
ing  in  greater  detail  protocol  tapes  from  a  previous  instructional  study  (Montague,  Riley,  A  Konoske, 
1985)  that  supplemented  the  standard  BEAE  course  with  instructions  to  think  about  current,  voltage, 
and  resistance  in  terms  of  a  'Chips*  analogy.  2  The  analogy  was  hypothesized  to  facilitate  the 
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acquisition  of  the  structural  knowledge  by  making  circuit  constraints  more  salient  and  by  providing 
subjects  with  simple  procedures  for  taking  those  constraints  explicitly  into  account.  It  was  also 
hypothesized  that  subjects  would  be  able  to  map  their  understanding  of  circuit  constraints  from  the 
analogy  to  their  understand  the  same  constraints  expressed  as  equations. 

Preliminary  results  indicate: 

•  Subjects  performed  quite  well  on  both  quantitative  and  qualitative  problems  when  they  used 
the  analogy. 

•  Subjects  had  difficulty  envisioning  solutions  to  some  problems,  forcing  them  to  work  out  the 
analogy  on  paper  and  pencil. 

•  Subjects  frequently  preferred  to  use  equations  to  solve  problems,  unless  prompted  to  use  the 
analogy. 

•  When  subjects  relied  on  equations,  they  frequently  made  errors  of  the  kind  observed  in  previ¬ 
ous  studies. 


Thus,  it  seems  that  the  Chips  analogy  does  facilitate  performance  but  mapping  knowledge  of 
constraints  from  the  analogy  to  circuit  equations  is  a  subtle  and  complex  issue  that  requires  consider¬ 
able  additional  empirical  and  theoretical  work.  We  plan  to  conduct  additional  empirical  studies  to 
determine  in  greater  detail  the  knowledge  students  acquire  from  instruction  using  the  Chips  analogy, 
and  under  what  conditions  they  are  able  to  transfer  that  knowledge  to  solve  problems  using  formulas. 

Croup  Data 

Dr.  Montague  administered  diagnostic  tests  to  BE/E  students  who  had  completed  the  instruction 
on  DC  circuits  and  passed  the  standard  tests.  Some  of  the  students  had  also  successfully  completed 
advanced  instruction  on  AC  circuits.  The  diagnostic  tests  included  questions  about  qualitative 
changes  similar  to  those  used  in  the  protocol  studies.  We  focussed  initially  on  qualitative  problems 
since  these  problems  are  the  quite  difficult  for  students.  The  tests  were  administered  in  three 
separate  studies: 

•  Study  1:  Effect  problems  only,  46  students. 

•  Study  2:  Cause  problems  only,  77  students. 

•  Study  3:  Effect  and  Cause  problems,  50  students. 


2.  The  analogy  involve*  a  ttack  of  chip*  and  one  or  more  dotted  boxes  connected  by  wire.  The  hose*  correspond  to  reaiitor* 
and  the  number  of  dot*  in  a  bos  determine*  it*  resistance  value.  The  stack  of  chip*  corresponds  to  the  applied  voltage  and  the 
amount  of  applied  voltage  is  specified  by  the  oumber  of  chip*  in  the  stack.  For  resistor*  io  *erie*.  current  is  determined  by 
•imply  distributing  the  chip*  of  totd  voltage  equdly  among  dl  the  resistor  dots;  the  number  of  chips  in  any  dot  corresponds  to 
the  current. 
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We  have  begun  analyzing  patterns  of  correct  responses  and  errors  for  individual  subjects.  In  all 
three  studies,  proportions  of  errors  were  quite  high,  sometimes  approaching  lJOO.  These  findings  indi¬ 
cate  that  even  when  students  perform  successfully  on  the  standard  tests  administered  in  BE&E,  they 
may  lack  an  understanding  of  important  circuit  concepts.  The  types  of  errors  made  by  subjects  in  the 
three  groups  were  similar  to  those  made  by  subjects  in  the  protocol  studies  which  gives  us  additional 
insight  into  the  nature  of  students*  difficulties. 

Theoretical  Analyses 

During  the  first  two  quarters,  considerable  time  was  spent  becoming  more  familiar  with  the  Dan¬ 
delion.  Initial  attempts  to  implement  computer  simulation  models  using  LOOPS  rules  and  knowledge 
structures  proved  time-consuming  and  unsatisfactory.  Development  of  the  computer  simulation 
models  is  now  progressing  smoothly  using  PRISM  which  we  obtained  at  the  end  of  July  from  Pat 
Langley.  We  have  agreed  to  serve  as  a  test-site  for  PRISM  and  help  debug  existing  problems  and 
implement  new  features.  Eric  Hestenes  has  just  completed  a  menu-interface  for  PRISM  that  greatly 
facilitates  the  retrieval,  display,  and  modification  of  production  rules  and  memory. 

Following  is  a  discussion  of  the  three  main  areas  of  focus  in  our  theoretical  analyses. 

Levels  of  Understanding  Constraints  in  Direct-Current  Circuits 

The  current  focus  of  the  theoretical  analyses  is  to  implement  Models  1  and  2  (described  in  Riley, 
1984)  in  PRISM.  The  differences  in  the  knowledge  included  in  Models  1  and  2  was  hypothesized  to 
represent  important  differences  in  understanding  circuit  relations.  In  addition,  the  models  will  be 
extended  to  include  the  additional  knowledge  required  to  solve  problems  involving  power,  opens,  and 
shorts.  Students  have  considerable  difficulty  solving  these  problems  (ci.,  Montague,  Riley,  & 
Konoske,  1985),  yet  understanding  these  relations  is  important  for  later  performance  in  BE&E  and 
eventually  for  maintaining  and  repairing  electronic  devices.  It  is  hoped  the  analyses  will  help  identify 
the  nature  of  students’  difficulties  and  suggest  possible  ways  to  improve  instruction. 

Once  implemented,  Model(l)  will  be  extended  further  to  include  a  mechanism  that  will  enable  it 
to  acquire  the  knowledge  in  Modcl(2),  the  model  that  represents  improved  understanding  of  impor¬ 
tant  circuit  relations.  A  similar  mechanism  was  developed  by  Greeno  (1983)  in  his  analysis  of  mean¬ 
ingful  learning  of  geometry  proof  problems.  Our  analysis  will  provide  a  better  understanding  of  the 
generality  of  this  learning  mechanism  and  show  how  acquiring  new  schematizing  production  plays  an 
important  role  in  bootstrapping  conceptual  understanding  during  learning. 

Towards  a  Theory  of  Errors 

Another  current  focus  of  our  theoretical  analyses  is  the  development  of  a  generative  theory  of 
errors.  In  both  the  protocol  studies  and  group  studies,  we  have  accumulated  large  numbers  of  error 
patterns  for  the  Effect  and  Cause  problems.  Of  course,  simply  accumulating  error  patterns  is  not  very 
interesting,  nor  very  useful.  Therefore,  wc  are  working  to  relate  those  error  patterns  to  a  smaller  set 
of  specific  deficiencies  in  students'  underlying  knowledge  structures.  So  far  we  have  developed  a  set 
of  30  degraded  production  rules  that  is  sufficient  to  generate  most  of  the  error  patterns  on  the  Effect 
questions.  Our  eventual  goal  is  to  develop  a  principled  account  for  how  degraded  rules  are  acquired 


Structural  Understanding  in  Problem  Solving 
August  30, 1985 


1st,  2nd  Quarter  1985  Progress  Report 

6 


and  applied  during  problem  solving.  For  example,  the  30  degraded  productions  were  generated  from 
Model(l)’s  correct  rules  by  systematically  omitting  conditions  that  specify  values  of  variables  that  must 
stay  the  same  for  a  production  to  apply.  Our  hypothesis  is  that  omitting  'same*  constraints  may  be  a 
quite  general  learning  mechanism  that  is  appropriate  in  many  contexts  but  leads  to  predictable  errors 
in  basic  electricity. 

Learning  by  Analogy 

A  model  will  be  developed  to  simulate  solving  basic  electricity  problems  using  the  Chips  analogy. 
Analyses  will  be  conducted  to  determine  how  the  knowledge  included  in  this  model  can  be  applied  to 
the  understanding  and  solution  of  the  same  problems  using  formulas.  The  analysis  will  draw  on 
Resnick,  Greeno,  and  Rowland’s  (1980)  formal  analysis  of  how  children  applied  their  understanding 
of  the  subtraction  procedure  using  Dienes  blocks  to  the  standard  subtraction  procedure  using  the 
written  number  symbols.  An  important  theoretical  question  in  this  context  is  how  a  learning  system 
identifies  the  properties  that  should  be  incorporated  into  a  new  procedure  that  is  formed  on  the  basis 
of  a  procedure  in  another  domain. 


Publications  and  Technical  Reports 

Riley,  M.  S.  (May  1985).  User  Understanding.  Technical  Report  ICS-8504,  Institute  for  Cognitive  Sci¬ 
ence,  University  of  California,  San  Diego.  NR667-538. 

Unpublished  Manuscripts 

Hcstenes,  E.  (July  1985).  The  role  of  previously  learned  concepts  and  perceptual  processes  in  learning 
basic  electricity.  Unpublished  senior  project  report.  San  Diego:  University  of  California,  San 
Diego. 

Leeman,  E.  (June  1985).  Chips  analogy.  Unpublished  senior  project  report.  San  Diego:  University  of 
California,  San  Diego. 

Moore,  R.  (June  1985).  Causal  relationships  in  transitional  mental  models.  Unpublished  senior  project 
report.  San  Diego:  University  of  California,  San  Diego. 

Prince,  L.  Structural  understanding:  Using  a  concrete  model  to  understand  basic  electricity  relationships. 
Unpublished  senior  project  report.  San  Diego:  University  of  California,  San  Diego. 
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Presentations 

Riley,  M.  S.  Structural  understanding  in  problem  salving.  Talk  presented  at  the  ONR  Contractors' 
Meeting,  University  of  Washington,  6  •  8  May,  1985. 

Riley,  M.  S.  Analyses  of  problem  solving  in  basic  electricity:  Implications  for  the  design  of  an  intelli¬ 
gent  tutor.  Talk  presented  at  the  workshop  on  'the  Bite-Sized  Architecture'  sponsored  by  the 
Intelligent  Tutoring  Systems  Group,  Learning  Research  and  Development  Center,  University  of 
Pittsburgh.  12  •  13  July,  1985. 


Conferences 

Mary  Riley  and  Eric  Hestenes  attended  the  Cognitive  Science  Society  meeting  at  the  University  of 
Irvine  (August  15  - 17). 

Eric  Hestenes  attended  the  International  Joint  Conference  on  Artificial  Intelligence  at  the  University 
of  California,  Los  Angeles  (August  18  -23). 
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